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ABSTRACT 

Image  segmentation  is  one  of  the  most  important  steps  involved  in  performing  higher  level  image  processing, 
e.g.  Medical  Imaging  (locate  tumors,  tissue  volumes  etc.),  Face  recognition,  Video  surveillance.  A  lot  of  researches  have 
dedicated  to  this  field  because  of  its  intrinsic  dilemma,  but  there  still  remain  a  wide  range  of  shortcomings  in  the  current 
segmentation  methods.  If  used  in  case  of  satellite  imagery  which  contains  tremendous  data  volume  and  very  multifarious 
ground  feature  distributions,  it  will  encounter  even  more  difficulties  in  extracting  meaningful  and  valuable  patterns. 

In  this  paper,  satellite  imagery  is  classified  into  two  types:  the  gray  value  and  surface  imagery,  and  then  look  for 
suitable  segmentation  methods.  Various  segmentation  algorithms  are  implemented  that  are  also  illustrated  and  validated 
with  typical  applications  on  segmenting  and  extracting  objects  of  interest  from  the  images. 
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INTRODUCTION 

Image  segmentation  is  one  of  the  most  important  operations  used  in  the  fields  of  computer  vision,  digital  signal 
processing  and  pattern  recognition  [1].  It  converts  original  pixel -based  imagery  into  more  condensed  and  abstract  forms 
which  helps  in  pattern  recognition,  segmentation  is  time  and  again  looked  as  the  first  step  for  other  higher  level  image 
interpretation,  e.g.  Medical  Imaging,  Video  surveillance  etc.  Depending  on  their  pixel  value  distributions,  imagery  is 
generally  divided  into  two  types,  gray  value  or  surface  imagery. 

Methods  which  are  suitable  for  gray  value  images  include  different  kind  of  histogram  threshold  methods,  edge 
detection  and  spatial  domain  based  methods  (e.g.,  region  growing,  split-and-merge  [2]).  Methods  used  for  surface  imagery 
use  statistical  methods  (e.g.,  autocorrelation,  filtering  template),  signal  processing  methods  (e.g.,  wavelet  transformation 
[4],  Gabor  filtering  [5])  and  model  based  methods  (e.g.,  Markov  and  Gibbs  random  fields,  fractal  models  etc). 

Even  though  the  development  of  image  segmentation  spans  more  than  several  decades  in  which  thousands  of 
methods  were  proposed,  a  single  method  is  suitable  for  all  kinds  of  imagery,  and  vice  versa.  It's  because  of  its  intrinsic 
predicament,  and  the  main  problems  involved  in  current  image  segmentation  methods  are. 

•  Inaccessible  from  the  vision  mechanism.  While  research  progresses  on  human  vision  mechanism,  current 
segmentation  methods  are  still  isolated  from  the  use  of  human  eye,  as  a  result  of  which  more  robust,  mechanism 
needs  to  be  implemented  for  real  time  and  parallel  segmentation  mechanism. 

•  No  human  knowledge  guidance.  There  will  be  inconsistent  interpretations  if  only  gray  values  or  spatial  context 
information  is  used  in  segmentation.  Humans  use  very  little  knowledge  beyond  the  image  itself  to  segmentation 
image  with  their  eyes.  The  guidance  of  prior  knowledge  is  extremely  important,  that  helps  us  to  reduce  the  image 
uncertainty,  and  improves  the  quality  and  efficiency  of  auto-segmentation.  Its  unrealistic  to  search  for  an 
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all-purpose  segmentation  method.  Currently  focuses  should  be  concentrated  on  finding  suitable  methods  for 
suitable  imagery  which  makes  it  issue-oriented. 

SATELLITE  IMAGE  SEGMENTATION 

Due  to  large  data  size  and  complicated  ground  feature  distributions,  it's  often  more  difficult  to  segment  remotely 
sensed  imagery.  The  Multi-band  property  of  satellite  imagery  causes  many  segmentation  methods  suitable  for  single  band 
imagery  inapplicable;  large  volumes  cause  the  computation  complexity  to  rise  rapidly  which  is  unreachable  by  many 
methods  designed  only  for  small  imagery. 

Besides,  too  complicated  ground  feature  distributions  of  satellite  imagery  also  make  it  uneasy  to  segment  or 
extract  meaningful  patterns  with  common  image  segmentation  methods. 

In  order  to  introduce  current  state-of-the-art  image  segmentation  methods  into  the  field  of  information  extraction 
from  satellite  imagery,  my  efforts  include  the  follows. 

•  A  lot  of  step-by-step  segmentation  wizards  are  designed  and  implemented  to  guide  the  users  in  their  segmentation. 
As  I  have  mentioned,  not  even  one  method  is  suitable  for  all  kinds  of  imagery.  With  these  wizards,  guiding 
knowledge  about  the  image  and  alternative  methods  are  presented  to  the  users,  which  help  them  to  choose  a  best 
method,  set  the  parameters,  and  iteratively  get  their  satisfying  results. 

•  Introduce  prior  knowledge  in  segmentations.  One  of  the  specificities  of  human  visual  segmentation  is  that  it 
involves  a  lot  of  prior  knowledge  which  is  also  why  it  often  gets  better  results  than  computer  programs. 
In  this  paper,  I  try  to  introduce  domain  knowledge  into  segmentations,  which  is  supposed  to  improve  the 
intelligence  of  the  methods.  For  example,  the  selection  of  segmentation  scales,  clustering  numbers,  and  the 
parameters  of  the  pre,  post-processes  might  be  specified  by  users  with  our  step-by-step  wizards. 

•  Multi-scale  integrating  framework  of  segmentation  methods  is  designed  and  implemented.  Multi-scale  is 
commonly  an  intrinsic  characteristic  of  many  natural  and  social  phenomena.  We  thus  need  to  develop  multi-scale 
segmentation  methods  or  integrate  methods  into  a  multi-scale  way  to  extract  interesting  spatial  patterns.  Scale 
space  theory  gives  us  a  theoretic  foundation  for  multi-scale  segmentations.  According  to  this  theory,  people  begin 
their  visual  segmentation  of  a  scene  with  large  scales,  which  indicates  that  their  visual  cells  first  catch  the  stimulus 
of  large  objects  or  background,  form  their  skeletons,  and  then  focus  step  by  step  on  the  details  or  sub-targets. 
In  this  paper,  I  try  to  implement  the  multi-scale  segmentation  framework  starting  from  discriminating  large  spatial 
patterns  with  suitable  methods  to  obtain  the  global  spatial  context,  which  will  then  help  to  extract  more  detailed 
features  with  other  segmentation  methods. 

INTEGRATION  OF  METHODS 

In  this  paper,  I  have  tried  to  integrate  current  state  of  art  segmentation  methods  into  a  unified  multi-scale 
segmentation  structure  to  bring  forth  their  respective  advantages.  Such  structure  will  help  to  select  suitable  methods  for 
suitable  targets. 

Besides,  with  different  image  resolutions,  different  ground  objects  segmented  in  different  scales,  large  scale 
objects  might  offer  spatial  contextual  knowledge  for  segmentations  in  other  finer  scales  Figure  1. 
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Gray-level  segmentation 
methods  -  Fuuy  threshold, 
Max  entropy  (each  identifying 
different  targets) 


Gray-level  andtexture 
segmentation  methods  - 
2DOtsu,  Markov  random  field, 
Gabor  filtering  (Different 
targets  identified) 


Shape  based  segmentation 
methods -Various  Edge  based 
segmentations,  Hough 
Transform  (detecting 
edges/lines) 

Figure  1:  An  Integration  Structure  of  Segmentation  Methods 

Diverse  Methods 

For  intensity  images  (images  comprised  of  point-wise  intensity  levels)  there  four  popular  segmentation 
approaches:  edge-based  methods,  region-based  techniques,  threshold  techniques,  and  connectivity-preserving  methods. 

Threshold  techniques  make  decisions  based  on  local  pixel  information  and  are  effective  when  the  intensity  levels 
of  the  objects  fall  squarely  outside  the  range  of  levels  in  the  background.  As  spatial  information  is  ignored,  blurred  region 
boundaries  may  cause  a  lot  of  problems. 

Edge-based  methods  use  contour  detection.  They  have  problems  in  connecting  together  broken  contour  lines  and 
hence  are  prone  to  failure  in  the  presence  of  blurring. 

A  region-based  method  partitions  an  image  into  connected  regions  by  grouping  neighboring  pixels  of  similar 
intensity  levels.  The  adjacent  regions  are  then  merged  under  some  criterion  involving  perhaps  homogeneity  or  sharpness  of 
region  boundaries.  A  rigid  criterion  will  create  fragmentation  whereas  lenient  ones  overlook  blurred  boundaries  and 
overmerge. 

A  connectivity-preserving  relaxation-based  segmentation  method,  generally  referred  to  as  the  active  contour 
model,  starts  with  an  initial  boundary  shape  represented  in  the  form  of  spline  curves,  and  then  iteratively  modifies  it  by 
applying  various  shrink/expansion  operations  according  to  some  energy  function.  Even  though  the  energy-minimizing 
model  is  not  new,  coupling  it  with  the  maintenance  of  an  "elastic"  contour  model  gives  it  an  interesting  new  twist.  As  usual 
with  such  methods,  getting  trapped  into  a  local  minimum  is  a  risk  against  which  one  must  guard;  this  is  no  easy  task. 

In  order  to  validate  the  structure,  I  have  already  implemented  more  than  6  segmentation  methods  including  Hough 
transformation,  Otsu2D,  Gabor  filtering,  improved  max  entropy  [6],  Markov  random  field  model  [7] [8],  etc. 
With  comparisons  on  segmenting  a  lot  of  satellite  imagery,  limitations  and  capabilities  of  these  methods  can  be  found  out. 
Table  1  and  Figure  2  are  one  of  the  experimental  results  of  segmenting  a  pearl  harbor  image  obtained  from  a  satellite. 


Large  scale 
targets 
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Table  1:  Methods  Used  for  Segmenting  Different  Harbor  Images 


Method 

Time 
Consumed 

Results 

Performance  Evaluation 

2D  Otsu 

5  seconds 

Figure  2  (a) 

Target  is  separated  with  speckles 

Max  entropy 

6  seconds 

Figure  2  (b) 

Target  is  partly  separated 

Markov  random  field 

9  seconds 

Figure  2  (c) 

Target  is  separated  from  background 

Gabor  filtering 

18  seconds 

Figure  2  (d) 

Segmentation  with  coarse  result 

Fuzzy  threshold  with 
genetic  algorithm 

22  seconds 

Figure  2  (e) 

The  main  target  is  partly  separated  with 
speckles 

Moment  method 

37  seconds 

Figure  2  (f) 

The  main  target  is  separated  with  coarse 
result  and  many  speckles 

(d)  (e)  (f) 

Figure  2:  Segmentation  Methods  Implemented 

ILLUSTRATION  ANALYSIS 

Large  Scale  Target  Segmentation  Using  Max  Entropy  Method 

With  the  guidance  of  the  proposed  integrating  structure,  Max  entropy  method  is  applied  to  segment  a  harbor  from 
a  high  spatial  resolution  satellite.  Because  it  is  only  a  simple  gray  value  threshold  method  but  with  relatively  fast  speed,  it 
can  get  good  results  in  large  scale  regional  segmentation.  If  successful,  it  could  further  be  applied  as  a  pre-processing  of 
other  finer  scale  segmentations,  providing  the  global  spatial  contextual  knowledge.  In  this  experiment,  I  have  tried  to 
separate  land  and  water  and  iteratively  separate  the  sands  from  the  land  with  inputs  of  different  number  of  classes. 

The  segmentation  results  in  Figure  3  illustrates  that  land  and  water  are  successfully  discriminated,  with  some 
errors  of  classifying  the  land  with  dark  tones  into  water.  This  seems  to  be  unavoidable  if  without  any  further  auxiliary 
processing.  With  more  number  of  classes,  the  sands  (shoals)  are  totally  separated,  with  some  mistakes  of  some  water  with 
relatively  bright  tones  are  classified  into  sands.  It  can  be  found  that  max  entropy  method  can  segment  large  regions  very 
fast  speed  as  well  as  keep  edges  of  targets  and  background.  Although  with  its  deficiencies,  max  entropy  method  is  not  bad 
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in  segmenting  this  kind  of  gray  value  satellite  imagery.  It  can  be  used  as  a  starting  method  for  the  initial,  large  scale 
segmentation  of  imagery  to  get  global  spatial  contextual  knowledge. 


(a)  (b)  (c) 

Figure  3:  Segmentation  of  Harbor  a)  Experimental  Area  b,  c)  Segmentation 
Depending  on  Number  of  Classes  Considered 


Middle  Scale  Target  Segmentation  with  Gabor  Filtering 

Gabor  filtering  segments  an  image  with  creating  a  group  of  filtering  templates  to  extract  features  with  different 
orientations  and  scales  and  then  classify  them.  For  images  with  sufficient  oriental  textual  information,  it  can  get  relatively 
good  results.  Figure  4  illustrates  some  results  of  extracting  some  middle  scale  targets,  residential  areas  in  Yanqing,  Beijing 
from  the  Beijing- 1  imagery. 

To  balance  the  efficiency  and  precision  of  this  algorithm,  12  texture  features  are  extracted  with  3  scales, 
4  orientations,  and  the  features  are  classified  into  2  classes.  Figure  4(b),  4(d)  show  the  texture  segmentation  capability  of 
this  method  for  middle  scale  targets. 


(a)  (b)  (c)  (d) 


Figure  4:  Segmentation  of  Residential  Areas  [10]:  a,  b)  Experimental  Area  1  and  its  Segmented  Image 
c,  d)  Experimental  Area  2  and  its  Segmented  Image 

CONCLUSIONS 

Image  segmentation  is  a  key  and  difficult  task  for  image  interpretation  with  a  lot  of  problems  unresolved. 
This  paper  introduces  some  work  dedicated  to  this  field,  which  includes  the  efforts  to  involve  visual  expertise  knowledge 
in  segmentation,  integrate  current  state  of  art  methods  into  a  universal  structure  to  adapt  to  different  applications,  etc. 
Although  with  some  progresses,  the  studies  are  just  getting  started.  More  attentions  can  be  applied  to  the  deeper,  seamless 
integration  of  geographical  knowledge,  multi-scale  segmentation  structure,  and  parallel  segmentation  strategy  to  notably 
increase  the  effect  and  efficiency  of  segmentation  methods  for  information  mining  from  satellite  imagery. 
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